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Abstract — This paper Investigates design of noncooperative 
games from an optimization and control theoretic perspective. 
Pricing mechanisms are used as a design tool to ensure that 
the Nash equilibrium of a fairly general class of noncooperative 
games satisfies certain global objectives such as welfare maxi- 
mization or achieving a certain level of quality-of-service (QoS). 
The class of games considered provide a theoretical basis for 
decentralized resource allocation and control problems including 
network congestion control, wireless uplink power control, and 
optical power control. 

The game design problem is analyzed under different knowl- 
edge assumptions (full versus limited information) and design 
objectives (QoS versus utility maximization) for separable and 
non-separable utility functions. The "price of anarchy" is shown 
not to be an inherent feature of full-information games that 
incorporate pricing mechanisms. Moreover, a simple linear pric- 
ing is shown to be sufficient for design of Nash equilibrium 
according to a chosen global objective for a fairly general class 
of games. Stability properties of the game and pricing dynamics 
are studied under the assumption of time-scale separation and in 
two separate time-scales. Thus, sufficient conditions are derived, 
which allow the designer to place the Nash equilibrium solution 
or to guide the system trajectory to a desired region or point. 
The obtained results are illustrated with a number of examples. 



I. Introduction 

Game theory has been used extensively as a quantita- 
tive framework for studying communication networks and 
distributed control systems among its other applications in 
engineering and economics. Game theoretic models provide 
not only a basis for analysis but also for design of network 
protocols and decentralized control schemes HI, ||2|- Despite 
widespread use of (noncooperative) game theory in engineer- 
ing, there is surprisingly little work on how to design games 
such that their outcome satisfies certain global objectives. 

While there is a general agreement on the usefulness of 
game theory, the issues of price of anarchy or efficiency loss 
associated with noncooperative games even under the exis- 
tence of pricing mechanisms, have been the subject of many 
investigations |J3l-|[5l. Consequently, different pricing schemes 
have been proposed in the literature aiming to improve Nash 
equilibrium (NE) efficiency with respect to a chosen criterion 
in specific settings 121, E-ISI- The research community has 
revisited the issue of game design again very recently ifTOl . 
ifm . These studies are limited either to special problem 
formulations or adopt specific efficiency criteria such as the 
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"system problem" of lfT2l . A related early line of work focuses 
specifically on three agent (player) dynamic noncooperative 
games with multi-levels of hierarchy ifTSl . lfT4l . where it 
has been shown that the leader has an optimal incentive 
policy which is linear in the partial dynamic measurement 
and induces the desired behavior on the two followers. In 
addition, a separate but substantial literature exists under the 
umbrella of implementation theory, especially in the field of 
economics, which focuses on finding fundamental bounds for 
games where the outcome satisfies some given criteria ifTSJI . 
These works, however, are often not algorithmic and do not 
have an engineering perspective. 

The main contribution of this paper is a treatment of 
game design from an optimization and control theoretic per- 
spective. A fairly general class of games are considered, 
which have been applied to a number of settings including 
network congestion control, wireless uplink power control, and 
optical power control lfT6l - ll20l . This paper -to the best of our 
knowledge- constitutes one of the few efforts aiming to inves- 
tigate the general problem of game design in a constructive 
manner /rom an optimization and control theoretic perspective 

While it is straightforward to optimize NE according to 
some criterion under full information, the problem is much 
more complicated under information and communication con- 
straints. The game or system designer (Figure [T]) usually does 
not have full information about the system parameters such 
as user preferences or utility functions. Under this kind of 
information constraints, the designer either deploys additional 
dynamic feedback mechanisms or requires side information 
from the system depending on the specific design objectives. 
An example for the former case is a dynamic pricing scheme 
operating as an "outer feedback loop". If the objective is 
to achieve a social optimum (e.g. maximization of sum of 
user utilities) or satisfying some quality of service (QoS) 
conditions, then the designer often needs limited but accurate 
(honest) information from users or the system. 

It is important to note that if the users have the capability of 
manipulating such side information, then the design problem 
can be more involved or even ill-defined. For example, the 
goal of reaching a social optimum without knowing true user 
utilities but having only access to manipulated data may not 
be a realistic or even feasible one ifTSJI . Although mechanisms 
such as VCG have been proposed to circumvent these issues, 
the resulting schemes are often limited and demanding in terms 
of communication requirements [1P|. As a starting point, the 
treatment in this paper is restricted to a class of games where 
players do not manipulate the game by deceiving the system 
designer Such players are sometimes called ''price-taking" 
as opposed to manipulative or "price-anticipating". Thus, the 
utility functions are assumed here to accurately reflect player 
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Game ^ 
Designer 




Fig. 1. The rules or pricing mechanisms within a game can be set by a 
"designer" to influence the outcome. 

As an additional contribution, this paper analyzes dynamic 
systems arising from game formulations and the general case 
of non-separable player utilities. In such cases of coupled util- 
ity functions, the utility of each player is affected by decisions 
of other players. The control theoretic approach adopted here 
based on game dynamics provides a more realistic model for 
a number of applications compared to a static optimization 
one. Stability properties of game and pricing dynamics are 
investigated under the assumption of time-scale separation and 
in two separate time-scales using Lyapunov theory and singu- 
lar perturbation approach. The sufficient conditions derived, 
which ensure system stability, are illustrated with examples. 

The rest of the paper is organized as follows. The next 
section introduces the underlying game model and problem 
formulation. Section |lll] studies static design of games. In 
SectionlTVl a dynamic control of games is discussed. SectionlV] 
discusses game design under incomplete information followed 
by a numerical simulation in Section [VT] The paper concludes 
with remarks of Section IVIII 



II. Game Model and Problem Formulation 

Consider a generic class of N player static noncooperative 
games, denoted by QO, on the action (strategy) space 51 C M^. 
The actions of players, who are autonomous decision makers, 
are denoted by the vector x G fl, with Xi being the i*^ player's 
action. Furthermore, the i*'' player is associated with a smooth 
(continuously differentiable) cost function, J^ : K x O ^ M, 
Ji{ai,x), i = 1,2, . . . ,N, parametrized by a scalar "pricing" 
or game parameter a^ G R. In some formulations, there may be 
(coupled) restrictions on the domain of these parameters such 
that a E ft C M^. Assuming a set of sufficient conditions 
for the existence of at least one Nash equilibrium (NE) are 
satisfied, define a game mapping, T (an inverse game mapping 
T) that maps game parameters a to NE points (NE points to 
parameters): 
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fl and T : il 
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such that 



= r(a*) and a* =f{x*) 



(1) 



(2) 



for any NE point x* and corresponding parameter vector a* . 
Notice that the mappings T and T are highly nonlinear. 



often not explicitly expressible, and may not be one-to-one 
or invertible except for special cases, i.e. games with special 
properties. 

We now define a class of games, Ql, by assuming a cost 
structure of the form 



Ji{ai,x) = ajpi(x) - Ui{x), 



(3) 



where a^ > Vi, the continuously differentiable functions 
Pi and Ui are convex and strictly concave with respect to Xi 
for any given .T_i, respectively. Let in addition the strategy 
space Vt to be convex, compact, and nonempty. Thus, a game 
belongs to class Ql, if the following assumptions are satisfied: 

Assumption 1. The strategy space Vt is convex, compact, and 
has a nonempty interior, fl ^ 0. 

Assumption 2. The cost function of the i*'' player Ji in dij 
is twice continuously differentiable in all its arguments and 
strictly convex in Xi, i.e., d'^Ji{x)/dxf > 0. 

Then, under the Assumptions [T] and |2] the game Ql admits 
a Nash equilibrium from Theorem 4.4 in IT] p. 176]. 

Next, we consider a specific class of games, Q2, as a special 
case of Ql with additional conditions on the cost structure, 
such that they admit a unique NE solution for each given a. 
Toward this end, define the pseudo-gradient operator 

VJ := [dJi{x)/dxi ■ ■■dJN{x)/dxNf := q{x). (4) 

Subsequently, let the N x N matrix Q{x) be the Jacobian of 
q{x) with respect to x: 
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where bi and a^ are defined as bi 



a-iN^ 



bN 

d^Jijx) 
dx'f 



(5) 



and Ui 



Q^.g^. , respectively. The following assumptions have to be 
satisfied in addition to Assumptions [T] and |2] in order for a 
game to be in class Q2: 

Assumption 3. The symmetric matrix Q{x) + Q{x)^, where 
Q{x) is defined in ^, is positive definite, i.e. Q{x) + Q{x)'^ > 
for all X £fl. 

Assumption 4. The strategy space fl of the class of games 
Q2 can be described as 



0:={xeR^ : hj{x) < 0, .? = 1, 2, ...r}, 



(6) 



where hi 



pTV 



-> R,j ~ 1,2, ...r, hj{x) is convex in 



its arguments for all j, and the set fl is bounded and has 
a nonempty interior In addition, the derivative of at least 
one of the constraints with respect to Xi, {dhj{x)/dxi, j = 
1, 2, . . .r}, is nonzero for i ^ 1, 2, . . . N, \fx G Vl. 

If Assumptions [Till] are satisfied, then the class of games Q2 
admits a unique Nash equilibrium. Furthermore, this unique 
NE can be an inner or a boundary solution. We refer to Ap- 
pendix as well as ||T|, 12111 . ll22ll for the details and an extensive 
analysis. Notice that a large set of network games belong 
to this class with notable examples of network congestion 



games H, 12311 . power control games in wireless networks lfT6l 
and optical networks fl9l. 

Assume that the utility function Ui accurately reflects the 
preferences of users or players, who are autonomous (in- 
dependent) decision makers. Then, the pricing function pi 
and parameters a are in this setting the only tools for the 
system designer to influence (optimize or control) the game 
outcome in order to achieve a desired objective. This task 
described in Problem [T] is similar -in spirit- to the goal of 
implementation theory or mechanism design in the economics 
literature ifTSJI with the important difference of not allowing 
users to knowingly manipulate the system, i.e. assuming price- 
taking users only. This game design problem is illustrated in 
Figure |2] 




Fig. 2. Game design involves controlling the game dynamics such that the NE, 
I* S n is moved to a feasible desired region fi n H or a specific optimal 
point X. 



Problem 1. How to choose the pricing function p and param- 
eters a such that the NE of games of class Q\ satisfies some 
desirable properties? 

Two specific but common examples of such properties are 
1) The NE coincides with the solution of a global opti- 
mization problem, e.g. welfare maximization: 



argmax\^ Ui{x) such that x e Sllj 



2) The NE satisfies some system or user-dependent con- 
straints such as capacity constraints, non-negativity, or 
performance bounds. For example, the favorable set Ct, 
can be defined as 

A := {x e M" : Xi > VJ, ^ x, < C, s,{x) > s, \/i}, 

i 

where C is a capacity constraint, s(-) is a quality of 
service (QoS) measure such as signal-to-interference 
ratio (SIR), and s denotes the minimum acceptable QoS 
level. 
An important aspect of Problem [T| is the amount of knowl- 
edge available to the system designer in optimization of NE. If 
there is complete knowledge of player preferences and global 
system objective, then the approach to be adopted can be quite 
different from the one when the designer has very limited 
information. In cases when the game dynamics are very fast, 
it is appropriate to focus on static optimization of the NE point. 

'All summations in the paper are from 1, . . . , N unless explicitly defined. 



Then, the actions of the system designer are assumed to be on 
a slower time-scale than the actual game dynamics between 
the players resulting in a hierarchical structure. This scenario 
is discussed in the next section. 

III. Static Design of Games 

When the game designer has a way of estimating game 
parameters, the game design can be posed as a static opti- 
mization problem, especially when the game dynamics are 
very fast and the actions of the system designer are on a slower 
time-scale than the actual game dynamics between the players. 
In contrast. Section |IV] presents a dynamic control approach 
appropriate when the game dynamics are slow or there are 
external disturbances. Then, the game is treated as a control 
system that needs to be stabilized around a desired point. 

When is it feasible to design a game such that the NE point 
can be located by the system designer to a point or region with 
desirable properties? In the point case, let target point be x. 
Then, the problem is to find an a such that a = T(x), for any 
desirable feasible x. The following surprisingly simple result 
addresses this problem for a fairly general class of games. 

Theorem III.l. For games of class Q2 with the cost structure 
given in (|5]) and under complete information assumption, ajfine 
pricing of the form, ap{-), is sufficient to locate the unique 
NE point of the game to any desirable feasible point, x £ 51, 
as long as 

%^ ^ 0, v. 
oxi 

Proof: The proof immediately follows from the first order 
necessary optimality conditions of player cost optimization 
problems due to the convexity of the cost structure and 
uniqueness of NE. 



dpi{x) dU.i{x) 
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= 0^ a,= 



dpi{x) 



dxi 



dU^jx) 
dxi 



Vi 



for any feasible x. ■ 

Remark 1. Theorem IIII. II can easily be extended to the case 
where users actions are on a multi-dimensional subspace if the 
users utility function is separable. 

Notice that even a simple linear pricing function p{xi) = Xi 
satisfies the conditions of the theorem and is sufficient for NE 
optimization. In this case any a; G il is feasible. However, 
a symmetric pricing scheme, where ai = aj yi,j, is not 
sufficient in general. As other examples, for p{xi) = e^' any 
X is feasible, while for p{xi) = xf any x ^ is feasible. 

If a game admits multiple NE, e.g. games of class Ql, 
then reaching a single desirable point does not make much 
sense. Furthermore, the problem of locating all NE points 
to a desirable region can be rather complex. Such cases can 
be handled either by exploiting any special structure of the 
game due to its specific problem domain or using numerical 
methods. 

It is interesting to note that due to the form of the player cost 
structure in ([3]), i.e. additive linear pricing, the optimization 
problem of individual players in the game can be interpreted 
as a variant of Legendre transform 1241 . In other words, given 



the other players' actions, the utility function best response 
of a player is transformed to a best response function which 
takes the pricing parameter as argument. 

Although tragedy of commons or price of anarchy are 
unavoidable in "pure" games without any external factors, 
they can be circumvented when additional mechanisms such 
as "pricing" are included in the game formulation. In parallel 
to some earlier results IZSl . Theorem IIII. II clearlv establishes 
that "loss of efficiency" or "price of anarchy" are not an 
inherent feature of a fairly general class of games with built- 
in pricing systems. If there is sufficient information, then any 
game of class Q2 can be designed through simple (e.g. linear) 
pricing mechanisms in such a way that any desirable criteria 
such as welfare maximization or QoS requirements are met at 
the unique NE solution. 

Example 1 

In order to illustrate the underlying issues in game de- 
sign problems with accurate but limited information, con- 
sider a noncooperative game formulation of the single-cell 
spread-spectrum uplink power control problem in wireless 
networks 1261 . Specifically, the game is played in a single 
cell with N mobiles competing for quality of service in terms 
of signal-to-interference ratio (SIR) and the base station acts 
as the game designer This can be formulated as a class Q2 
game with the following general cost structure 



For notational convenience, we define the matrix 



Ji{ai,x) = aiPi{x) ~Ui{x), \/i, 



(7) 



where the objective is to locate the NE to a region that satisfies 
some feasibility and QoS constraints. Further assume that the 
utility function Ui of user i in this game is 



U,{x) =/3,log(l + s,;(x)), 



(8) 



where sAx) 



Z^j^^ it'iXj 



Here, s represents the SIR with hi > Vi denoting channel 
gain parameters and ct^ a noise term. Then, each user i decides 
on its own power level Xi and is associated with the cost 
function Ji in (jTji. 

The desired region for the NE of this game could be shaped 
by feasibility constraints such as positivity of user actions 
(here uplink transmission power levels) and an upper-bound 
on the sum of them, and/or some chosen minimum SIR levels 
(assuming these are chosen such that the region is not empty). 
A concrete example region Q, can be defined as 

f2 := {a- G M^ : a-j > 0, s,{x) > s, Vi}, (9) 

where Si are user-specific minimum SIR levels. A detailed 
analysis of an example case is provided next. For a separate 
but similar example of this formulation we refer to ll20l . 

As a special case, the pricing function Pi{x) is chosen next 
to be linear in Xi such that 

Ji{ai,x) = aiX-i ~ /3jlog(l + s,(a;)), 

where Si{x) is defined in (H). Under appropriate assumptions, 
the game is one of class Q2 and admits a unique inner NE 
solution, X* . 
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Then, the NE is the solution of 

Ax* = 



Lh 



N 



hN \ 
Lhi 

hN 

Lh2 

hN 
Lh. 



1 ; 



(10) 



where 



c := 



ai 



a 
Thi 



Pn_ 

OiN 



Lh 



N 



The desired QoS region f2 in (|9]l can be alternatively 
described in terms of received power levels at the base stations 
and in matrix form ll27l : 



O = {.T e M^ : .T, > Vi, Sx > b}, 
where the matrix S is defined as 
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-hiS2 
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(11) 



and 



Sicr 



L 



If the designer, here the base station, has full information, 
then given a feasible target SIR level s it is possible to solve 
for a pricing vector a such that the NE is on the boundary 
of Cl, i.e. Sx = b. This is due to both matrices A and S 
being nonsingular as hi > Vi. Hence, the appropriate pricing 
vector a can be immediately obtained from the boundary 
solution 

c = AS~\h), 

and the definition of c. 

IV. Dynamic Control of Games 

Many games are solved in a distributed manner Hence, 
the convergence of the system trajectory to the equilibrium 
point may not be very fast and the time-scale separation 
between system designers actions and actual game dynamics 
may fail. Then, the game design can be modeled as a feedback 
control system which utilizes pricing as the control input 
and the desired target as the reference (see Figure [3]i. This 
formulation also brings a certain robustness with respect to 
initial conditions or game (system) parameters. The latter case 
is especially relevant for systems that are non-stationary over 
longer time periods and can also be formulated as a tracking 
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Fig. 3. Feedback control of the game (NE, x") using pricing a as tlie control 
parameter and x as the desired reference signal. 



problem. Specific examples are congestion and power control 
game formulations, ||9l, HSl, 1231 . 

The counterpart of the feasibility question in the case of 
static game optimization relates in the dynamic control setting 
to the controllability of the system shown in Figure [5] or 
reachability of a state x. In order to provide a concrete example 
to the problem of controllability, consider a game of class Q2 
where the players adopt a gradient algorithm to optimize their 
own cost. Then, the game dynamics are; 



dJi{x) dUi{x) dpi{x) 



ai Vi, 



(12) 



dxi dxi dxi 

where a acts as the feedback control on the outcome of the 
game. Here, the objective is to investigate the conditions under 
which the game system is controllable. We write (fT2b in vector 
form as 

N 

i = /(a;)+^g,(a;)a, = /(:r)+G(x)« (13) 

where a = [ai . . . aj^Y" , the matrix 



G{x) = [gi{x),...,gN{x)], 



and the vector 



/(x) 



dUi{x) 



dU.ix) 



dxi dxi 

Alternatively, the matrix G is given by 

dpi{x) 



dUNJx) 
dxN 



Gix) 



dxi 







dpijx) 
dxi 



dxN 



Based on the standard theorem on controllability using Lie 
brackets 1281 Chapter 1], we obtain the following result. 

Theorem IV.l. For games of class Q2 with cost structure (O 
and game dynamics ( 1721 ). or ( 1731 ). a sufficient condition for 
local reachability around a point xq is that the distribution C 
satisfies the rank condition at Xq, dimC{xo) = N where 



C=[g,,... 
where [f,gi\{x) 



dgi{x) 



[f,9i 
df{x) 



, ad)gi 



a /(^) S — 9i{x) is the Lie 

ox ax 

bracket of f and gi, and ad^gi denote higher order Lie 

brackets defined recursively by adHgi{x) = [f,ad"f~ gi]{x). 

Remark 2. Notice that if the diagonal matrix G{xo) has rank 
N, any x locally around xq is reachable in finite time under 



piecewise constant input functions, which is equivalent to the 
feasibility condition in Theorem IIII.ll In addition, for the 
simple linear pricing function p(a-i) ~ xi any x is immediately 
reachable since G{x) is constant and invertible. 

Alternatively, the problem can be posed as one of asymp- 
totic set point regulation, i.e., to find a feedback control of 
the form a ~ Q;(x,a;) such that the system trajectory x (and 
eventually the NE) converges to the desired reference point 
X ll28l Chapter 8]. Considering the game system (fT3] l and the 
change of variables x ~ x — x, m the new coordinates the 
game system becomes 



X ~ f{x + x) + G(x + x)a 



(14) 



and the design problem is to find control a to stabilize the 
equilibrium x = 0. The first necessary condition, which 
translates into feasibility condition for x is that (fl4] i has an 
equilibrium at the origin, i.e. there exists a steady-state control 
as = c{x) that solves 



Q = f{x) + G{x)c{x) 



(15) 



The component as is the first component in a needed to main- 
tain equilibrium at the origin, while a second component is 
needed to asymptotically stabiUze this equiUbrium in the first 
approximation. Thus, based on the necessary and sufficient 
conditions for asymptotic regulation problem (see Theorem 
8.3.2 in |f28l), speciaUzed for constant reference, a feedback 
control law for the pricing parameter that solves this problem 
in the full information case is a = a{x^ x) with 



a(x, x) = c{x) -V K (x ~ x) 



(16) 



where c[x) solves ( fTSl l and K selected such that (A + _B K) 
has eigenvalues with negative real part. The matrices A and 
B denote the Jacobians of / and g at the origin, respectively. 
Using (fT3T l it can be seen that the feasibility condition ( fTSb 
is equivalent to feasibility in the static case (Theorem IIII.ll l. 
Also, for G(x) full rank this feasibility condition of the regu- 
lation problem corresponds to x being reachable (see remark 
after Theorem II V. lb . Note also, that for constant reference x, 
integral control could be included in the design formulation 
by augmenting the system ( fT3] l with a stack of N integrators 

b^x (17) 

The task is to design a stabilizing feedback controller a ~ 
aix^a) that stabilizes the augmented state model ( I14I17I ) at 
the equilibrium point (0, Cs) , where Us produces the desired 
steady-state control 



Example 2 

Consider the power control problem in optical networks 
lfT9l with linear pricing and an optical signal-to-noise ratio 
(OSNR)-like utility function. Similar to the wireless case, the 
players here compete for quality of service in terms of OSNR. 
and are coupled due to interference. The user i chooses own 
power level xi based on the following cost function 



Ji{x) = UiXi - I3i log 1 + 



1 -^iili{x) 



(18) 



a special case of (O, where /?;, ai are design parameters and 
7,; {x) denotes the OSNR 

Xi 



lt{x) 



no 



E,r, 



(19) 



for a given system matrix F and input noise iiq. Let f be the 
matrix with elements 



Oi, l = J 

Then the corresponding dynamic system, similar to ( fT3] l. is 
where /(x) is the vector with elements 



(20) 



dxi 



_ij i ijXj 



no + J2i 
or, in shifted coordinates, x ^ x — x, 

X = f(x + x) — a 
Linearizing around the origin, x = {), yields B 

df{x) 



-I and 



A 
with f as in ( l20b and 

D = diag 



dx 



^ DT 



-a. A 



[na+Y^j^ij^jf 



An appropriate gain K can be selected to ensure stability 
at the origin for A + BK if the feasibility conditions are 
satisfied, ll28l . i.e., {A,B) is stabilizable and 
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DV -I 
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is nonsingular, which is clearly the case. Thus, tracking is 
achievable, and pole placement methods can be applied to 
derive an exact gain parameter K. In addition, the system 
can be made more robust by applying integral control as an 
alternative. The same two feasibility conditions that apply to 
feedforward control, which our system {A, B) already satis- 
fies, also apply to integral control. The augmented linearized 
system (A, B) with integral control has the form 
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with control law a = Kx + Kia. The gains K and A'/ can 
be determined using pole placement applied to the augmented 

system. 

V. Game Design under Information Constraints 

Unlike the case discussed in the previous section, the 
system designer usually does not have full information about 
the system parameters such as user preferences or utility 
functions. Under such information constraints, the designer 
either deploys additional dynamic feedback mechanisms or 
requires side information from the system, depending on the 
specific design objectives. An example for the former case is 



a dynamic pricing scheme operating as an "outer feedback 
loop". If the objective is to achieve a social optimum (e.g. 
maximization of sum of user utilities) or satisfying some 
QoS constraints, then the designer often needs accurate side 
information from users or the system. Assuming that users are 
non-manipulative (honest or price-taking) and given accurate 
side information, the task of the designer can be formulated as 
an optimization problem even if it is solved indirectly or in a 
distributed manner. Next, an example formulation is provided 
that illustrates the underlying concepts. The objective is let the 
NE coincide with a social optimum (maximizing sum of user 
utilities) in the general case of non-separable user utilities of 
the form Ui{x). 

A. Pricing Dynamics under Time-Scale Separation 

Define a strictly concave and smooth social welfare function 
U{x) which is a sum of concave and non-separable utility 
functions U{x) := 'Yl,iUi{x) and admits a global maximum 
X = argmax-c ^j f/i(a:). As before the cost function is 
Ji{ai,x) = aiPi{x) — Ui{x). Here, unlike the separable 
one in 1291 , side information (e.g. Ui{x*)) is required in 
order to bring the NE to the social maximum point. The 
social maximum is defined easily via the first order optimality 
conditions 



where 



m 

dx 



ix) 



dx 



dU, 



(x) = 0, 



^^ dxi ^ 



E. 



dxN 



ix) 



The social maximum is shown to coincide with the unique 
equilibrium (and NE) of the following pricing mechanism 



ttj 



^ dxi da. 



If these pricing dynamics are on a slower time scale than 
the game dynamics, then the system designer can obtain 
sufficiently accurate estimates of dUi{x*)/dxi and dx*/dai. 

As one possibility, if the users adopt a gradient algorithm, 
then the designer can use past values of x* and a along 
with the exact form of the pricing functions p to estimate 
dUi{x*)/dxi directly without requiring any side information 
(except from some fixed system parameters). In addition, side 
information (e.g. Ui{x*)) is required to estimate dUi{x*)/dxj 
and dx* /dttj for all i, j. 

Assume an ideal case where the parameter estimation is 
perfectly accurate. Then, the pricing mechanism above ensures 
that the NE point of the underlying game globally asymptot- 
ically converges to the maximum of the social welfare func- 
tion|3 The next theorem follows from Lyapunov theory and 
LaSalle's theorem in a straightforward manner. The Lyapunov 
function is chosen to be the negative of social welfare function, 
V = -U. 

Assume that the pricing mechanism is on a slower time 
scale than the actual game dynamics leading to a time-scale 



For simplicity, the social maximum point is implicitly assumed to be on 
the solution space of the game. 



separation, and hence to a hierarchically structured problem. 
Assuming this time-scale separation, for simplicity, initially 
only the pricing dynamics (slower dynamics) is considered. 

Theorem V.l. Consider a class Q2 game with cost struc- 
ture di]) and user utilities Ui \fi, where the objective function 
U{x) := X)i^i(^) odtnits a unique inner global maximum 
X ~ argmaXj: W(x). Then, the pricing mechanism (|27} en- 
sures that the NE point of the underlying game, x*, globally 
asymptotically converges to the maximum of the social welfare 
function, x, if the Jacobian matrix of the mapping T with 
respect to pricing vector a, defined as 



dx* 
H{a) := —(a) 



dxl 



(a) 



hJ 



,iV, 



is non-singular. 



Proof: The pricing dynamics (slow) are analyzed, assum- 
ing that the user dynamics is fast and converges quickly to 
X* = T{a) for any given a. The pricing scheme ( l2TT i admits a 
unique equilibrium if and only if dU{x)/dx ~ Oo In order to 
prove this statement, first change the summation order in (ISTT i 
and write it in vector form as 



H^{a) 



S<^) 



(22) 



where x = T(a). The sufficiency immediately follows from 
non-singularity of H. If dU/dx = 0, then d = and at the 
same time the system is at the social maximum x. Hence, at 
the unique equilibrium of the pricing scheme, d, the social 
maximum is achieved. Next, the necessity is shown under the 
same condition. At the unique equilibrium point a of the pric- 
ing scheme (|22] |. it is necessary that dU /dx = due to non- 
singularity of of H. Consequently, at the unique equilibrium 
point a of the pricing scheme the objective function U{x) 
necessarily reaches its maximum x characterizing the social 
maximum, hence x = T{a) or d = T{x). 

In order to establish convergence of ( l2Tl i. define the Lya- 
punov function 



V{a) 



-U 



MTia)) 



on the compact game domain il and a G M.^ . Taking 
the derivative of V with respect to time along the pricing 
dynamics iTii yields 



V 






^^ dx 



EJE.E 



J dak 

dUi dxi 



^ dxj dak 



ak 



j:ki<^kr<o. 



Thus V = Q only at dj = 0, Vj, or at its unique equilibrium. 
Hence, by the LaSalle's theorem, (Theorem 4.4, ll30l ). the 
pricing scheme (|2TI) globally asymptotically converges to its 
unique equilibrium at which the NE solution coincides with 
the social maximum. ■ 

^We drop in the rest of the proof the (■)* notation, which characterizes the 
NE, for convenience. 



In the special case of separable utility functions U{x) 
Ei f^i(^i)' the pricing mechanism simplifies to 



E2^5iv^, 



dx* dai 



(23) 



which ensures that the NE point of the underlying game 
globally asymptotically converges to the maximum of the 
social welfare function under the same condition. 



Example 3 

Consider a game with separable utility functions with the 
cost 

Jj(ai,x) = tti {J2i^i) - Ui{xi), 

where Ui := (ii log(l + x^) -kiXi). 

This type of utility function may arise due to inherent physical 
constraints on player actions such as battery constraints on 
uplink transmission power levels in wireless devices. Then, 
the NE solutions is 



A: 



1. 



Notice that, the matrix H{a) is diagonal in this case. Further- 
more, we can explicitly find 



dai 



A 



{ui + k,y 



<o, 



from which non-singularity of H immediately follows. The 
properties of this example also hold for a quadratic pricing 
function replacing the linear one, i.e., pi{x) = J^i^f- How- 
ever, for pi{x) = e^i^S X* is not independent of aj and 
non-singularity of H{a) is not immediate. 



Example 4 

Consider a variant of Example 1 for the non-separable utility 
case by defining a linear pricing and a signal-to-interference 
ratio (SIR)-like utility function 



Ji{ai 



/^ 



where 



Si{x) :-- 



Xi -/3jlog(l + Si{x)), 



riiXi 



T,j^ihjXj+a' 



Here, the parameter L = 1 for simplicity. Then, it follows 
that under non-singularity conditions on the system matrix A 
defined in ( fTOl i. the unique NE is given as x* ~ A^^ v where 
V = [vi], Vi ~ — — a. In this case x* depends on all pricing 
parameters, a, and H{a) is not diagonal. However, we can 
still explicitly find dx* /da, and it can be shown that, under 
non- singularity conditions on A'^ , H is non-singular 



Example 5 

An example of dynamic game control is obtained by ex- 
tending Example 1 to the limited information case where the 
designer does not have access to user preferences. Then, a 
dynamic a pricing mechanism can be deployed. Toward this 
end, define a set of penalty functions pi{xi) to bring the system 
within the desired region 



Pi{xi) ■■-' 



/(bj - {Sx)t), if Sj < Si 
0, else 



(24) 



where the scalar function /(•) is smooth and nondecreasing 
in its argument, and 

/(O) = /(O) = 0. 

For example, / could be a quadratic function. 
A possible pricing function is then 



^ dx* dai 



Vi. 



(25) 



It is assumed here that the designer (base station) has access 
to system parameters L, h, and a. The other terms can be 
estimated through iterative observations fiV\. 

Finally, this pricing mechanism ensures that the NE point 
of the underlying game, x* , enters the desired QoS region Vt. 
To show this, define the Lyapunov function 



y := 






Pi{.Xi) 



on the compact game domain il. Taking the derivative of V 
with respect to time along the pricing dynamics dZSl l yields 



dpi 



dxi 



V = -V. -^ V. --i 
■^^i a™ ^-^1 a„. 



dxi 



'^ da, ^' 
dpi dxj 

dxi duj 

with V < outside the set Ct and ^ = if and only if 
cti ~ Vi. Hence, the system converges to the desired region 
fj under the pricing mechanism. 

B. Game and Pricing Dynamics on Two Time-Scales 

The previous analysis has focused on pricing dynamics 
under the time-scale separation, where game dynamics are 
assumed to be sufficiently fast. Removing this assumption 
for a complete treatment, two loops (on two time-scales) 
will be considered: one outer loop for pricing (d;) and an 
inner loop for actions {xi). The next result presents the full 
analysis taking both user and pricing dynamics into account 
and is based on a singular perturbation or boundary layer 
approach, ISOll . Towards this end. Theorem IV. II is extended to 
analyze both pricing (slow) dynamics and user (fast) dynamics 
by using a combined Lyapunov function. 

Theorem V.2. Define an objective fiinction lA{x) := 
"^iUii^x) which admits a unique inner global maximum 
X = aigu\ax.xU{x) under suitable assumptions for user 
utilities Ui Vi in a class Q2 game. Then, under the pricing 



mechanism ( 1271) the user dynamics ( 1721 ) globally asymptoti- 
cally converges to the maximum of the social welfare function, 
X, if the two systems are on separate time-scales, the Jacobian 
matrix of the mapping T with respect to pricing vector a. 



Hia)^'£ia) = 



dx* 



I,] 



1,.. 



.N, 



is non-singular and the Jacobian matrix Q of Tr~> ( 1721) . with 

OXi 

respect to x 

Q{a,x) 



d'^J^{a,x) 



dxjdxi 



i,j = l,...,N, 



is positive definite. 

Proof: Consider the pricing dynamics jTH or ( l22b on 
the slow time-scale t, and the user dynamics ( fT2b or ( fT3] l on 
the fast time-scale, tf. Here t = etf, with e > a small 
scaling parameter. The equilibrium of the full system ( fTSl i. ( l22b 
is described by x = T{a) and a = T'ix)- In singular 
perturbation form the full system is written in the shifted 
coordinates {a, x) as 



a 



Tf- 



dUjx) 
dx 



ux 

f{x + x) + g[x + x) (a + a) 



(26) 



where x ^ x — x, a ~ a — a, a = da/dt and the origin 
(a, x) — (0, 0) is its equilibrium. 

In the shifted coordinates the slow manifold is defined as 
X = h{a) by taking e = and solving on the right-hand side 
of (|26ll, i.e., 

= f{x + x) + g{x + x) {a + a) 

Since = f{x) + g{x) a for x = T{q), the foregoing yields 
X + X = T{a + a), so that x = h{a) for h{a) defined by 



h{a) :— T{a + a) 



T{a + d) - T{a) (27) 



with /i(0) = 0. 

Once the slow manifold is determined, a solution that starts 
in the manifold is known to stay inside for all times. In order 
to analyze the behavior in the neighborhood of this manifold, 
an additional change of variables is used y = x — h{a) on 
the fast dynamics x with respect to the slow manifold. Using 
this change of variables the full system (l26l ) is written in the 
(d, y) coordinates as 



H' (d + d) 



dU{x) 
dx 



(28) 



ey 



where y 



-y+T(a+a) 

f{y + T{a + a)) + g{y + r{a + d)) (d + d) 

T 



dh(a) ^^T,~ ~\ 
-e^-V^77^ (a + a) 



da 



dU{x) 



dx 



=y+T(a+a) 

h{a) and ( |27] l have been used to obtain 



X ^ y + T{a + a). 



On the slow manifold (for y = 0), the dynamics of a in 
define the reduced system, i.e., 

T 



a^ H^ {a + a) 



dU{x) 
dx 



(29) 



a:— 7~(a+a) 

On the fast tf time-scale, the dynamics of y in (|28] | define the 
boundary-layer system 

-17-^ f(y + T(^ + ")) + 3(2/ + Ti^ + ")) (" + ") (30) 

dtf 



dy 

IT' 



ey and e = has been used on the right-hand 



where 

side of ( BsT l. Using ( fT2] l. yields component-wise from (|30] | 

dy^ _ ^ dJi{a,x 
dtf ^ 



dxi 



(31) 



3:=j/+T(a+a) 

To analyze the behavior and stability properties of the full 
system ( l28T i, an overall composite Lyapunov function is 
defined based on Lyapunov functions for the reduced ( |29] | and 
boundary-layer systems, dSTT i. respectively. For the reduced 
system ( |29l ) the same Lyapunov function is used, i.e., 

V{a) = -U{T{& + a)). (32) 

As shown in the proof of Theorem IV. II the derivative of V 
with respect to time along the reduced system dynamics (|29] | 
is negative, 

V{a)<~\\af <Q (33) 

For the boundary-layer system ([3TJ the following candidate 
Lyapunov function is defined 

M^2(5,2/) = J^0^(5,y + r(5 + d)) (34) 



where 



4)i{a,y + T{a + a)) := 



dJi{a,x) 



dxi 



-y+T{a+a) 



Note that 0., = ^ and, by using (EB, 



d(l)i _ d'^y,, _ _\^ \ <-' 



d fdJi{a,x) 



dxi 



(35) 



The derivative of W2 with respect to time along the boundary- 
layer system trajectory (|30] | is given as 



dW'. 



2 , ~ 
a 



dtf 
or, by using (l35l l 



J 2/) = X!'^*("'^ + '^(" + "))^ 



''^^5,,)^-^5:<^.e 



di 



tJ'+'J 



where. 



©.,. 



dxjdxi 



evaluated at a; = y + T{& + a). In vector form this yields 



dW2 

an 



{a,y) = -(f Q(t>< 



(36) 



since > by assumption. Moreover, for any a in an 
appropriate domain around an upper bound that depends 
only on y can be found for (|36] |. 

Finally, using the Lyapunov functions for the reduced ( l32b 
and boundary-layer systems ( |34| |. the following candidate 
Lyapunov function is defined for the full singularly perturbed 
system ( |28] | 

W{a,y) = {l-d)V{a) + d W2 (5, y) 

for some < d < 1. Then based on ( l33l [36] l using standard 
arguments as in the proof of Theorem n.3 in |[30|, it can be 
shown that there exists an e* > such that for every < e < 
e*, VF is a Lyapunov function for the full system ( |28] ) and the 
system is asymptotically stable around the origin. ■ 

Remark 3. The bound and required interconnection relations 
follow immediately if, the stronger condition is imposed that 
the reduced and boundary-layer systems are exponentially 
stable as in Theorem 1L4 in 



Example 6 

Consider the Example 2 with cost Ji a special case of ([3]), 
for linear pricing and OSNR-like utility 



U,ix) 



A: (log 



1 



1 -Twyiix) 



x^) (37) 



Such an utility that may arise due to specific constraints on 
individual channel power ensures the existence of a unique 
maximum for U{x) = J^i Ui{x). The Nash equilibrium x* 
can be obtained from dJi/dxi = 0. If Oi are selected such 
that the following diagonal dominance condition holds 

a,>Y, r»j (38) 

then X* is explicitly expressed as 

X* ^f-^C{a) 



where 



r, 



y I 



1 = J 

and C{a) is a vector with elements ^°'_j^^. 

f-Miag 



, s dx* , , 
B[a) = — a 
da 



- uq. Hence, 



(39) 



is clearly non-singular. The second condition in Theorem IV.2I 
is that G > 0, where Q is for this example: 



e^ 



e, 



dx^ 
dx-jdx 



[no 



Z^j I ij^j 
^iPi^ ij 



j^^t {no + T,j'^ijXjy 



It can be immediately seen that if ( [38] ) holds, then the matrix 
Q is strictly diagonal dominant. If in addition, a^ > J^ij^i ^ji 
it can be shown that 8 is strictly diagonal dominant, hence 
is positive definite. The closed loop system ( I2TI ). ( fT2b is 



a = H^{a) 



dx ^ ' 



(40) 



m 



where H^(a) is defined in ( |39] |, and based on dJTJ i. the j 

dlA 
element in — — (x), for Uix) -.^ V. UAx), is given as 
ox 



(41) 



with 

X,{x) 



Ik^ 



ajPj 



"0 + Efc Tj-fca;*; 



/,(.T)-X,(a;) 



-ft- 



(42) 



E 



dpPpL pjXp 



p^j ('^0 + Efej^p rpfca;fc)(no + (Efe TpkXk) 

Thus ( |40l i. ( l4n i represent the closed-loop dynamic system. The 
objective function U satisfies d^U/d^x < for a nonempty 
set of the design variables a, /3, but the conditions are very 
complicated and as such the details are omitted. The following 
section presents an example based on realistic parameters for 
the two channel case. 
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Fig. 4. Channel prices, a, as a function of time. 



VI. Simulations 

The dynamic system described by (|40] | and W^ of Example 
6 in the previous section is simulated numerically in the two 
players case. Consider an optical fiber link with ten amplifiers, 
each with a parabolic gain shape according to the formula 
G = -4el6 X (A - 1555 x lO-^)^ + 15 dB, where A denotes 
a channel wavelength, and a span loss of 10 dB. The system 
matrix F is obtained as 

2.47 X 10-3 2.61 X 10-3 
2.36 X 10-3 2.5 X 10-3 



r = 



The following parameters are used: /3i = 1, a = [0.485, 0.48] 
such that they satisfy the diagonal dominance condition on 
r and d^lA/d^x < 0. These parameters yield the equilib- 
rium solution X = [0.0134, 0.0128] milliwatt (mW) and 
a = [73.4, 76.9]. An input noise power of no = 0.43 nanowatt 
(nW) is considered. The closed-loop system (|40] |. JAli is 
simulated for e = 0.01, i.e., on two time scales such that 
in discrete-time form the pricing algorithm ( l40l i is run every 
100 iterations of the user algorithm dTTT i. The user algorithm 
is implemented in a decentraUzed way such that only the 
explicitly measurable OSNR ji signal is fed back to the 
respective channel source. 

The simulations, which take a = [18.35 19.23] and 
X = [0.00043 0.00043]'^ as initial conditions and run for 50 
(5000) iterations of the pricing (users) algorithm, show a clear 
convergence to the equilibrium solution x, a and to the OSNR 
values of approximately 23dB in Figures |5]anc(6ll. The Figure|4] 
depicts the evolution of channel prices, a, as a function of 
time. 

VII. Conclusion 

The Nash equilibrium design of noncooperative games is 
discussed from an optimization and control theoretic perspec- 
tive. It is shown that for a fairly general class of noncoopera- 
tive games, pricing mechanisms can be used as a design tool 
to optimize and control the outcome of the game such that 
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Fig. 5. Channel powers x (mW) as a function of time. 
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certain predefined global objectives are satisfied. The class 
of games studied are applicable to a number of problems, 
such as network congestion control, wireless uplink power 
control, and optical power control. The game design problem 
is analyzed under full and limited information assumptions, 
perfect and imperfect time-scale separation between game and 
pricing dynamics, and both for separable and non-separable 
utility functions. Sufficient conditions are derived, which allow 
the designer to place the Nash equilibrium solution or to guide 
the system trajectory to a desired region or point. The results 
obtained are illustrated through a number of examples from 
wireless and optical networks. 

There are many directions for extending the results pre- 
sented. One of them is further application of game design 
methods to specific problems such as power control in optical 
networks. Others include secure communication and spectrum 
allocation in wireless networks. Yet another interesting direc- 
tion is the analysis of estimation error effects on mechanism 
stability for the limited information case. In addition, the 
stability of the combined pricing and game system can be 
analyzed under time delays, which often have a nonnegligible 
effect. 
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Appendix 

This appendix revisits the analysis in ||2T1 . Il22ll on existence 
and uniqueness of Nash equilibrium under Assumptions [T]|4] 

For games of class Ql and Q2, the existence of NE 
follows immediately from a standard theorem of game theory 
(Theorem 4.4, p. 176, in UJ) under Assumptions [T] and |2] 

In view of Assumption]?] the Lagrangian function for player 
i in this game is given by 



Li{x,^i) = Ji{x) +y^^fii,jhj{x), 






(43) 



where fXij > 0, j = 1, 2, . . . r are the Lagrange multipliers 
of player i 1331 p. 278]. We now provide a proposition 
for Q2 games with conditions similar to the well known 
Karush-Kuhn-Tucker necessary conditions (Proposition 3.3.1, 
p. 310, I33I)- 

Proposition 1. Let x* be a NE point of a Q2 game and As- 
sumptions\r^hold. There exists then a unique set of Lagrange 
multipliers, {(pij '■ j = 1, 2, . . .r, i ~ 1, 2, . . . N}, such that 



dL{x*,(j)) dJi{x*) 



dxi 



dxi 



E 



dhj{x*) 



0, 



f^jj 



> 0, Vi,j, and 



^■1,0 



dxi 

1^1,2, ...N, 
0, Vj^A,(x*),V*, 



where Ai{x*) is the set of active constraints in i^^ player's 
minimization problem at NE point x* . 

Proof: The proof essentially follows lines similar to the 
ones of the Proposition 3.3.1 of |[33l , where the penalty 
approach is used to approximate the original constrained 
problem by an unconstrained problem that involves a violation 
of the constraints. The main difference here is the repetition 
of this process for each individual Xi at the NE point x* . ■ 

Define now a more compact notation the vector of La- 
grangian functions as L := [Li, . . . , Lat], and the N x N 
diagonal matrix of Lagrange multipliers for the j*'' constraint 
as $j =diag[<?!)i,j,<?!)2,j,...(?!>N,j]. 

By Proposition [T] and Assumption |4] a NE point x^^^ 
satisfies 



VL(x(i),$(i)) 






(1)^ 



V/ij(a;^^0 = 0, (44) 



.(1) 



where $^ > is unique for each j. Assume there are two 
different NE points a;'°) and a;'^^-'. Then, one can also write the 
counterpart of ( l44l i for x'"'. Following an argument similar to 
the one in the proof of Theorem 2 in ||2T1 . one can show that 
this leads to a contradiction. We present a brief outline of a 
simplified version of that proof for the sake of completeness. 
Multiplying ( l44l i and its counterpart for a;*^*^' from left by 
(x*-*^^ — x^^'>)'^, and then adding them together, we obtain 

= (a;(0)-x(i))^(g(a:W)-(7(x("))) 

(g(a;(i))-9(a;(0)))^(x(0)-xW) 

(x(i)) 



(45) 



-$f V/i,(a;W)]. 



Define the strategy vector x{d) as a convex combination of 
the two equilibrium points x'"' , a;^^-' : 

e'a-(i) + (1 - 6l)a;(''\ 



x{e) 



where < 9 < 1. Take the derivative of q{x{9)) with respect 
to 0, 

^ = Q(.^W)^-QW^))(.«-^°^), (46) 

where Q{x) is defined in (|5]l. Integrating ( |46] | over 6 yields 



<7(.T(i')-g(a;(°') = 



Q{x{e))de 



(a;(i)-a;(°)). (47) 



Multiplying Wt\ from left by (a;^^-* — x^*'-')^, the transpose of 
(|47] | from right by (x^^^ — x^°^), and adding these two terms 
yields 



(x 



(1) 



.(0)-!^ 



Q{x(e)) + Q' {x{0))d0 



(x 



(i)-x(«)l 



(48) 

Since Q{x{9))+Q'^{x{9)) is positive definite by Assumption|3] 
and the sum of two positive definite matrices is positive 



definite, the matrix Q := J„ Q{x{9))+Q'^{x{9))d9 is positive 
definite. 

Similarly, we have 

'^^-Him)'-^-Hi.m^^^.% (49) 

where H{x) is the Jacobian of Vh{x) and positive definite 
due to convexity of h{x) by definition. The third term in (l45T l 



(x(o) - x(i))^E;^J$f V/i,(x(0)) - $;.'^V/i,(x(i))], 



is less than 






$f][/.,(x(i))-/.,(xW)], 



due to convexity of h{x). Since for each constraint j, hj{x) < 
Va;, $^*^/ij(a:(*)) = 0, i = 0, 1, and $j is positive 
definite, where the latter two follow from Karush-Kuhn-Tucker 
conditions, this term is also non-positive. 

The sum of the first two terms in (|45] | are the negative of 
( l48T l, which is strictly positive for all x^^-' =^ x^^\ Hence, 
( |45T l is strictly negative which leads to a contradiction unless 
2;(i) = j;(o) Thus, there exists a unique NE point in the class 
of games Q2. 
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